Skip to content

fix: Neo4j vector search pre-filter and metadata handling#1359

Merged
CaralHsi merged 6 commits intoMemTensor:dev-20260323-v2.0.11from
CaralHsi:feat/customize_llm
Mar 27, 2026
Merged

fix: Neo4j vector search pre-filter and metadata handling#1359
CaralHsi merged 6 commits intoMemTensor:dev-20260323-v2.0.11from
CaralHsi:feat/customize_llm

Conversation

@CaralHsi
Copy link
Copy Markdown
Collaborator

@CaralHsi CaralHsi commented Mar 26, 2026

Description

Fix Neo4j vector search returning empty results in shared-database multi-tenant mode, along with several related metadata handling bugs.

Core fix: Replace post-filter approach (db.index.vector.queryNodes + WHERE) with Neo4j 5.18+ pre-filtering (MATCH + WHERE + vector.similarity.cosine()). This ensures filters are applied before similarity computation, so target user's nodes are never excluded by a global top-k truncation.

Additional fixes:

  • metadata["sources"]metadata.get("sources") to prevent KeyError when sources is absent
  • _parse_node sources deserialization: [0] == "}"[-1] == "}" (was checking first char instead of last, making json.loads never execute)
  • neo4j_community.py add_node: gracefully skip vector DB insert when embedding is None instead of raising ValueError
  • neo4j_community.py add_nodes_batch: skip VecDBItem creation for nodes without embeddings

Related Issue (Required): Fixes #1360

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Unit Test
  • Test Script Or Test Steps (please provide)

Unit tests (6 tests, mocked Neo4j driver):

  • TestVectorSearchPreFilter: Verify pre-filter path uses MATCH + vector.similarity.cosine() when WHERE clauses present, and ANN path uses queryNodes when no filters
  • TestSourcesKeyErrorRegression: Verify add_node and _parse_node work without sources key

Integration tests (3 tests, real Neo4j 5.18+):

  • TestNeo4jPreFilterIntegration: Insert 50 nodes for other users + 3 for target user, verify search with user_name filter returns all 3 target user nodes

Run: pytest tests/graph_dbs/test_neo4j_vector_search.py -v

Checklist

  • I have performed a self-review of my own code | 我已自行检查了自己的代码
  • I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
  • I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
  • I have created related documentation issue/PR in MemOS-Docs (if applicable) | 我已在 MemOS-Docs 中创建了相关的文档 issue/PR(如果适用)
  • I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用)
  • I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人

Reviewer Checklist

@CaralHsi CaralHsi changed the title Feat/customize llm fix: Neo4j vector search pre-filter and metadata handling Mar 26, 2026
@CaralHsi CaralHsi marked this pull request as ready for review March 27, 2026 02:21
@CaralHsi CaralHsi merged commit fd2d4c7 into MemTensor:dev-20260323-v2.0.11 Mar 27, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant